Computer training system with audible answers to spoken questions

ABSTRACT

A phonics training system provides immediate, audible and virtual answers to questions regarding various images such as objects, animals and people, posed by a child when the child views such images on a video display terminal of the system. The system provides virtual answers to questions without the need for an instruction or teacher and includes a computer having a video output terminal and an electronic library containing common answers to basic questions.

RELATED APPLICATIONS

[0001] This application is a continuation-in-part of pending U.S.application Ser. No. 09/766,951, filed Jan. 22, 2001, which is acontinuation-in-part of Ser. No. 09/025,347, filed Feb. 18, 1998, nowU.S. Pat. No. 6,227,863, the entire contents of both applications beingherein incorporated by reference.

FIELD OF THE INVENTION

[0002] This invention relates to a system and method for self-teaching,including self-teaching of young children, including children havingdevelopmental disabilities. The system of this invention includesapparatus for sensing an audible word or command of a young child foreffecting a computer generated response to said audible word or command,wherein said response can include a graphical depiction of the lettersof said audible word or command, an object image related to said audibleword or command, or any combination thereof. In addition to thegraphical depiction generated by the computer response to audible wordsor commands, the system also provides audible answers preferably by avirtual teacher to questions orally posed by a child pertaining to thedepiction displayed.

DESCRIPTION OF THE PRIOR ART

[0003] The main purposeful activity undertaken by young children isplay. From a developmental perspective, play is often considered aspractice for the child's later roles in life. Through play a child canexplore the environment and gain knowledge regarding the physical lawsgoverning objects, socialize with adults and peers to gain importantinterpersonal skills, and use his or her imagination and begin honingthe pathways for conscious thought. Thus, it can be argued that play inhumans is a tool leading to perceptual, conceptual, intellectual andlanguage development, the basic building blocks required for theformulation of higher cognitive functions.

[0004] For healthy children, very early childhood play is eitheroriented toward physical objects or toward symbol manipulation.Meaningful verbalizations may not yet be possible. However, eventuallythrough play, the child begins to attach specific names to objects. Soonafterwards, more complex sentences are learned and the child talks tohimself or herself in order to achieve specific activities. Meaningfulself-speech, when activity and verbalization merge and the child can saywhat he or she is doing, is believed to be a significant point inintellectual development. Self-speech is eventually made non-verbal, andwe talk to ourselves through specific mental activities. This is oftenreferred to as self-reflective activity and occurs throughout childhoodand even adulthood. For self-reflective activity to occur, one must beable to recall and manipulate specific memory events. The brain systemthat this refers to is working memory which is theorized to have both averbal/language component, a phonological loop, and a visual imagerycomponent, the visuospatial sketch pad.

[0005] The child's brain is different from the adult brain in that it isa very dynamic structure that is evolving. A two year old child hastwice as many synapses (connections) in the brain as an adult. The youngbrain must use these connections or lose them. Thus, failure to learn askill during a critical or sensitive period has important significance.

[0006] It is known that the greatest period of intellectual developmentpotential for a child is before the age of three. However, children donot normally start any formal education until age five or six, andinfants typically in an ideal family only receive one to two hours ofdaily intellectual stimulation.

[0007] It would, thus, be advantageous to provide a virtual play andeducational environment for children, including those children whoselearning experiences have not been adequately developed, or for medicalreasons have been slow to develop. Such a virtual educationalenvironment would be advantageous for several reasons: developmentalmilestones may be achieved more quickly; the child may learn todistinguish between images and scenes created from multiple images; andit may give the child a vehicle for self-reflective activity and thus bea seed for the development of imagination, consciousness, andcommunication. One such virtual learning environment system is describedin U.S. Pat. No. 5,815,147 (to Bogen, et al. issued Sep. 29, 1998).

[0008] Notwithstanding the advances set forth in Bogen, et al, i.e.,virtual play environment, the potential of such environment has beenlimited by its focus on children with limited interactive capabilities,and the fact that interacting is preprogammed and not based on thechild's spontaneous behavior. Clearly the potential of such a system,which has yet to be more fully realized either by both children withspecial needs, or by children not similarly encumbered, has not beenfully realized because of such limited focus (children with learning orphysical disabilities). Thus, the Bogen, et al, system is inherentlylimited because of its failure to provide adequate interaction of thechild with the virtual play environment through the use of multipleinteractive input; and, a more varied or comprehensive set of learningobjectives (lessons) to retain the child's interest. Accordingly, therecontinues to exist a need for such enhancement so as to provide alearning environment wherein each child, including those with limitedphysical and/or emotional development, can cause the environment toreact with it in some meaningful and responsive way; and, encourage thechild to look beyond a possible limited learning environment to relatedsubjects in alternative sources of such information.

[0009] It is, therefore, an object of this invention to provide acomputer training system for self-teaching, particularly for youngchildren. A further object of the invention is to provide a virtualenvironmental system for those children whose learning experiences havenot yet adequately developed, or for medical reasons have been slow todevelop. Another object of this invention is to provide a trainingsystem for a child of pre-school age, which system can provide anaudible response to questions posed orally by such child with respect towords and/or images appearing on a video display terminal of suchsystem. Another object of this invention is to provide the depiction ofa virtual teacher on a video display terminal, who audibly answersquestions posed orally by a child using the system. These and otherobjects will readily become apparent to those skilled in the art inlight of the teachings herein set forth.

SUMMARY OF THE INVENTION

[0010] In its broad respect, this invention is directed to a computertraining system having one or more input and output devices foreffecting interactive communications with the learning system and avideo display terminal for graphical depictions of the system responseto the output devices, wherein the interactive communication is promptedby one or more user initiated inputs, the improvement comprising:

[0011] a) means for processing information generated by a user of thevirtual learning system;

[0012] b) means for displaying on said video display terminal,information generated by the virtual learning system in response to theinformation generated by the user;

[0013] c) means for the presentation in audible or video form by theuser inquiries regarding the information observed on the displayterminal;

[0014] d) an electronic library of answers to inquiries which commonlyarise from viewing the information; and

[0015] e) means for the recognition of words spoken in the user'sinquiry and rendering, via the output device from the library, theanswer most closely related to at least some of the works and theirsequences as presented in the inquiry.

[0016] While the self-teaching system of the present invention isparticularly useful for young children, it can, of course, findapplication in the studies of older students and even in training ofadults.

BRIEF DESCRIPTION OF THE DRAWINGS

[0017]FIG. 1 depicts the virtual learning environment system of thisinvention.

DESCRIPTION OF THE PREFERRED EMBODIMENT

[0018] The training system of the present invention is a virtuallearning environment system as shown in FIG. 1.

[0019] The system comprises a general-purpose programmable computer,such as personal computer 10, a visual display device 12 (preferably atouch screen visual display device), and appropriate input/outputdevices such as a microphone and/or speaker, a keyboard, a mouse notshown, and a video camera 22, and could also include a single switchinput device (joy stick) not shown. Computer 10 also has access to theInternet via Internet access port 23. Depending upon the age and theextent of the user's physical abilities, either the touch screen videodisplay device, microphone, the keyboard, mouse, or joy stick, or anycombination thereof, may be employed as one means for selecting andmanipulating objects shown on the display. In a preferred embodiment ofthe virtual learning environment illustrated herein, the computer 10 isprogrammed to provide a non-directed play environment for the user inthe form of real or simulated images, and an electronic library 24associated with the computer in which are electronically stored answersto commonly asked inquiries resulting from observation of images.

[0020] Referring now to FIG. 1, illustrated therein is an educationaltraining system in accordance with the invention that includes a digitalcomputer 10 having a central processing unit (CPU). The output ofcomputer 10 is coupled to a video monitor or terminal 11 having a screen12 on which may be presented the letters of the alphabet which spell aword spoken into the system by a pre-school child, an image of theobject identified by this word, or real or simulated images, visualimages generated by image software 16 and/or video camera 22. Also,computer 10 is coupled to an electronic library 24, having voiceresponse means for orally providing answers to questions posed throughvoice recognition unit 13.

[0021] In the input of computer 10 is a peripheral in the form of avoice recognition unit 13 coupled to a microphone 14. When a pre-schoolchild (or other speaker) speaks into microphone 14, unit 13 thenrecognizes the distinctive pattern of sounds in the voice of the child,and converts these sounds into digitized signals that are fed intocomputer 10 and processed therein.

[0022] Associated with computer 10 is an electronic phonetics dictionary20. Digitally stored in the electronic phonetics library are both thephonetic sounds which constitute the words contained in the dictionary.The contents of this dictionary are downloaded into the memory of thecomputer.

[0023] The voice recognition unit 13 is operatively coupled to theelectronic phonetics dictionary 20, so as to condition this dictionaryto render it responsive to the distinctive voice pattern of the childusing the system. Thus, while the phonetic sounds of the words digitallystored in the electronic dictionary are synthetically generated sounds,not sounds derived from a human voice, the dictionary must be able tocompare a word or phrase spoken by a pre-school child with the same wordor phrase stored in the dictionary. In order to be able to do so, beforethe child uses the system, he or she first speaks for a few minutes intothe voice recognition unit to permit the unit to analyze the voice ofthe child and then recognize its unique pattern.

[0024] The elctronic-dictionary 20 has digitally stored therein avocabulary of words and letters of the alphabet which spell each word.When a child speaks into the input of the computer, the computer scansthe words in the dictionary to find the stored digitized sounds of theword from the voice recognition unit 13.

[0025] When a match is found by the computer, then the letters or theword yielded in the output of the computer are presented on screen 12 ofthe computer terminal 11. Hence, the child can see how the word he hasjust spoken into the computer is spelled.

[0026] Also associated with computer 10 is an image software module 16,having stored therein a library of digitized images, each being apicture or drawing of an object identified by a respective word includedin the vocabulary of the electronic dictionary. Thus, if the dictionarycontains the word DOG, CAT, BOY, HOUSE and BOTTLE in the module 16,there will be a picture or drawing of each of these objects.

[0027] When computer 10 responds to an incoming word, such as CAT, andpresents on screen 12 the spelling of CAT, it at the same time presentson the screen an image of a cat.

[0028] Thus, whatever word is spoken into the input of the computer by achild using the system which identifies an object, this causes thecomputer to yield in its output terminal the letters spelling this word,and an image of the object identified thereby.

[0029] In this way, a pre-school child using the system is taught how tospell the words which are included in his vocabulary, while seeing foreach word an image of the object presented by the word. This serves notonly to impress on the child's mind the spelling of the word, but alsoserves to clarify its meaning. Thus, if a pre-school child is confusedas to the distinction between the words BOY and BOOK, when he says theword BOOK and then sees a book on the screen, he knows that a book isnot a boy.

[0030] In practice, the system need not be confined to words that arenouns and identify objects, for the system can-include verbs andadjectives commonly used by pre-school children, such as RUN, JUMP, BIGand LITTLE. In order to illustrate these verbs and adjectives, the imagesoftware 16 must include appropriate illustrations. Thus, RUN can beillustrated by a child running, BIG by a giant and SMALL by a midget.

[0031] One may use as images cartoon characters that most children arefamiliar with, having seen them on TV shows for children. Thus, PLUTO,the Disney character, can be used to represent a dog, and DONALD DUCK torepresent a duck.

[0032] It is also useful for teaching pre-school children to spell andread, to include arithmetic numbers such as the digits one to ten, andhow these digits are spelled. When children learn how numbers arespelled, they can read stories that refer to “three bears” or “fivechairs.”

[0033] In the learning process, it is desirable when a child is shownhow to spell a word, that he then be required to spell the word aloud,for in doing so, one then knows whether the child has learned thespelling.

[0034] The system may be modified to test the pre-school child todetermine whether he remembers how a word is spelled. Thus, after achild sees how the word CAT is spelled on screen 12, he may then berequested to speak into the microphone how this word is spelled bysaying the letter C, then the letter A, and finally the letter T.

[0035] In accordance with one aspect of the invention, there is alsoincluded video camera 22, which can, for example, be focused on aninfant in his or her crib. Movements from the infant can be detected bythe video camera, digitized and applied to computer 10. Those movements,which initially may be random, can be utilized to allow the infant tobegin to learn how to manipulate objects in a virtual environment. Forexample, if the infant initially moves his or her arm, this could resultin movement of an object in the virtual environment causing the objectto be moved from a first location to a second location depending on theparticular movement of the infant. As the infant would continue to makerandom movements, there would be provided feedback from video monitor11, which would indicate to the infant a positive indication of thecorrect movement to move a particular object in the virtual environment.In this manner, the infant would gradually learn the movements requiredto move objects in the virtual environment.

[0036] The invention also provides for a mediator to monitor the user'sprogress via connection to the Internet. For example, if the infant usersuccessfully accomplished a particular task, the system could bemodified to provide a new task for the user, or alternatively providefeedback to a caregiver on the progress of the infant user.

[0037] In addition, the connection to the Internet could also be used toprovide additional information to be displayed on video monitor 11.

[0038] In the teaching system shown in FIG. 1, a child sees on thescreen of the computer terminal the spelling of the word he has spokeninto the computer and an image of the object identified by this word.But, the system depends on the ability of the child to speak into thecomputer the many words he knows, yet there is nothing to spur the childto speak words he uses infrequently. Thus, it may not occur to the childto speak the word BEAR or COW, for he rarely has occasion to use thesewords even though he knows them.

[0039] In order, therefore, for the child to exploit all of the wordsphonetically stored in the electronic dictionary 20, the training systemmay include a random access selector which may be actuated by the childor a caregiver using the system.

[0040] When the child or a caregiver activates a random access,selector, it then chooses at random the stored digitized sounds of oneof the words in the dictionary, say BEAR, and feeds those sounds to ananalog-to-digital converter D/A. The converter yields an analog signalof BEAR. This signal is amplified in an analog amplifier, and thenreproduced by a loud speaker.

[0041] Thus, each time the child or a caregiver accesses the randomaccess selector he or she hears a word randomly selected from thephonics dictionary. Because the child does not know what word to expect,this lends interest to this operation. The child must then speak theword he has just heard into the computer input, so that he can now seehow it is spelled and what object it identifies.

[0042] Thus, the child may not have in his working vocabulary the wordBOAT. But, when he hears this word extracted from the dictionary andspeaks it into the computer, the child is then informed as to themeaning of this word by an image of a boat and how it is spelled.

[0043] In this way, the child is encouraged to explore the entireelectronic dictionary rather than only those words that come to mindwhen using the training system.

[0044] Alternatively, one may provide a dedicated computer in whosememory is stored the electronic dictionary and the library of digitizedimages to provide a self-sufficient and compact device for home use by apre-school child, or for use in an elementary school.

[0045] If, in the first grade of an elementary school, the students aregiven a simple story book to read, the dedicated computer in itselectronic dictionary can include all of the words in the book exceptfor articles and propositions. In this way, a child who masters thespelling of the words in the electronic dictionary will experience nodifficulty in reading the book.

[0046] A further unique feature of the aforementioned training systemallows a child of pre-school and elementary school age to actually posequestions to the system and receive a digitized oral answer to questionswhich a child of that age might be expected to ask after viewing animage on the video display terminal 11. The oral answer can be given onthe display terminal by a virtual teacher so that the young child mightbelieve he or she is hearing directly from a live person via thecomputer.

[0047] The child can access the library to obtain answers to commonlyasked questions by speaking the questions into the training system. Whenwords such as what, which, where, why, when, if, or other suitable wordsare spoken, the library is scanned for the words in the questions whichwhen taken together will select the closest answer to the question used.

[0048] It may also be feasible for an icon such as a question mark toappear on the display terminal when the child is using the computertraining system. When the child wishes to pose a question, he or she canclick on the icon, and have immediate access to the library or answersfor the spoken question.

[0049] The computer thereupon makes a determination of what is on thedisplay terminal and searches the electronic library for the appropriateanswer to the questions. Upon selection of the answer, it is conveyed tothe child electronically by a digitized voice given simultaneously witha image on the display terminal.

[0050] For example, a pre-school child in using the training system mayspeak the word CAR and observe the image of an automobile on the videodisplay terminal, with the letters C-A-R appearing, and a virtualteacher sounding the word CAR and also spelling aloud the letters C-A-R.

[0051] In viewing the image of the car on the display terminal, thechild may not know the word for the color of the particular cardepicted. By asking “what is the color of the car,” the library scansits data bank and finds the closest answer to the words WHAT; CAR; andCOLOR, and a virtual teacher appears on the display terminal screen andorally announces that the car shown is BEIGE, while both spelling andpronouncing the word one or more times.

[0052] A more mature child may want to know the year and model of thesame car, and pose the questions “what is the year and producer of thecar?” After scanning the library data bank, a dialog box appears and thevirtual teacher will announce that it is a 2001, Ford.

[0053] Hence, by using the present training system, the child can haveimmediate access to answers to questions which might enter his or hermind upon viewing an image on the video display terminal. With continueduse of the training system, the child will readily accept thepresentation of answers by a virtual teacher as though a human wereproviding the answers as each question was presented.

[0054] The training system of the present invention, therefore, providesa learning environment which in the mind of the child approximatesreality about what appears on the video display terminal. The child asksa question by speaking and receives an immediate audible answer from avirtual teacher. In view of the oral presentation of a question and theimmediate receipt of an oral answer, it is as if a human teacher werepresent in the room and available to answer the child's questions.

[0055] The advantage of using the system of this invention is that itdoes not require the presence of a teacher to answer commonly askedquestions by the user. Depending upon the complexity of the images ormaterial appearing on the video display terminal, most simple questionscan be virtually answered immediately after the question is posed.

[0056] The absence of a live teacher will, of course, be a great savingsof time and expense.

[0057] However, a virtual teacher appearing on the video display unitwith the answer to the inquiry will give the child the sense that his orher question is being answered by a teacher who actually is providingthe answer.

[0058] A further advantage is that the answer to a question appearsimmediately If a child cannot get an immediate answer, he or she mayforget to ask the question of a teacher at a later date and hencedeprive themselves of added knowledge.

[0059] While many of the tutorials which come with new software displayan animated character or person who audibly comments on whether theoperator has performed a function properly or not, there is no means forpresenting audibly or otherwise a question and receiving an audibleanswer from a virtual teacher.

[0060] A child using a training system which does not provide immediateanswers to questions that the child might have, may be reluctant orforget to raise the question at a later date. By obtaining an immediateanswer to his or her questions, the knowledge of the child is enhancedwhich otherwise may not occur until later, or forgotten by the child.

[0061] Although the invention has been illustrated by the foregoingdisclosure, it is not to be construed as being limited to the materialdisclosed therein, but rather, it is directed to the generic area ashereinbefore disclosed. Various modifications and embodiments thereofcan be made without departing from the spirit or scope thereof.

What is claimed is:
 1. A virtual learning system having one or moreinput or output devices for effecting interactive communications withthe learning system, and a video display terminal for graphicaldepiction of the system response to the input devices, wherein theinteractive communication is prompted by one or more user initiatedinputs, the improvement comprising: a) means for processing informationgenerated by a user of the virtual learning system; b) means fordisplaying on said video display terminal information generated by thevertical learning system in response to the information generated by theuser; c) means for presentation by the user in audible or video forminquiries regarding the information observed on the display terminal; d)an electronic library of answers to inquiries which commonly arise fromviewing the information; and e) means for the recognition of the user'sinquiry and rendering via the output device from the library, the answermost closely related to at least some of the words and their sequence aspresented in the inquiry.
 2. A virtual learning system in accordancewith claim 1, wherein said information generated by said user includesaudible information.
 3. A virtual learning system in accordance withclaim 2, wherein said information generated by said user includes videoinformation.
 4. A virtual learning system in accordance with claim 3,wherein there is further included means to monitor the educationalprocess of said user of said virtual learning system.
 5. A virtuallearning system in accordance with claim 4, wherein said monitor meansfurther includes means to modify said virtual system learningenvironment based on progress of said user.
 6. A virtual learning systemin accordance with claim 1, wherein said virtual learning systemprovides for access to remotely stored educational material which may beaccessed and displayed by said virtual learning system in response tothe educational progress of said user.
 7. A virtual learning system inaccordance with claim 1, wherein a virtual image of a human appears onthe video display terminal and is synchronized with the audiblepresentation of the answer from the electronic library.
 8. A virtuallearning system in accordance with claim 7, wherein said virtual imageof a human is a teacher.
 9. A virtual learning system in accordance withclaim 8, wherein said teacher appears in a classroom setting.
 10. Avirtual learning system in accordance with claim 1, wherein said virtuallearning system, in response to said audible or video information, willdisplay the questions posed by the user of the system.
 11. A virtuallearning system adopted to provide a child of limited knowledge withimmediate audible virtual answers to inquiries that the child may havewhen observing images on a video display terminal, the systemcomprising: a) a computer having an output terminal provided with anaudio speaker; b) an electronic library associated with the componentsin which is digitally stored answers to commonly asked inquiriesresulting from observation of images on the video display terminal; andc) means, including a voice recognition unit in the input of thecomputer, acting to recognize distinct words and their sequence patternof a child speaking into the unit, whereby certain words and theirsequence prompt the computer to scan the library to find the specificanswer most closely related to the inquiry and when a match is found,audibly presented to answer to the child.
 12. The virtual learningsystem in accordance with claim 11, wherein a virtual image of a humanappears on the video display terminal and is synchronized with theaudible presentation of the answer.
 13. The virtual learning system inaccordance with claim 12, wherein the human is a teacher.
 14. Thevirtual learning system in accordance with claim 13, wherein saidteacher appears in a classroom setting.
 15. The virtual learning systemin accordance with claim 1, wherein the presentation by the user is inaudible form.
 16. The virtual learning system in accordance with claim1, wherein the means for recognition of the user's inquiry is in wordrecognition.